Some analysis projects are better performed by expert humans (“experts”) than by computers. For instance, labeling website photos or selecting the most representative photos from a large database of photos are projects that experts are more effective at completing than computers. As another example web search companies can use experts to test the effectiveness of search engines and do quality control experiments. Experts can also be useful in, for example, writing product descriptions or splitting up audio files. Other categories where experts are more useful than computers include catalogue and data management, search optimization, database creation, and content management. However, using experts can be very expensive and time consuming.
One solution is to use a crowdsourcing service or system. Crowdsourcing services and systems provide online marketplaces where businesses and other entities can submit projects that are best completed by experts. Yet, instead of experts working on the projects, hundreds of online workers or “online non-experts” complete the projects. These crowdsourcing systems decrease the time and cost of a project while only marginally decreasing the work product as compared to what an expert could accomplish (and in some cases producing a better work product than what experts are capable of). The idea is that tens, hundreds, or thousands of non-experts can do the job of an expert for a fraction of the cost and in a fraction of the time. The businesses or other entities are called “requesters,” and the requesters submit projects to a website of any one of the crowdsourcing services.